Image acquisition device and method and system for acquiring focusing information for specimen

ABSTRACT

In an image acquisition device, an optical path difference generating member can form an optical path length difference of a second optical image without splitting light in a second optical path. This can suppress the quantity of light required for the second optical path to obtain information of the focal position, whereby a quantity of light can be secured for a first imaging device to capture an image. The image acquisition device synchronizes the movement of a predetermined part of a sample within a field of an objective lens with rolling readout such that each pixel column of a second imaging device is exposed to an optical image of the predetermined part in the sample.

CROSS-REFERENCE TO RELATED APPLICATIONS

This is application is a continuation application of U.S. patentapplication Ser. No. 14/786,576, filed Oct. 23, 2015, which is a § 371national stage application of International Application No.PCT/JP2014/061180, filed Apr. 21, 2014, which claims the benefit ofJapanese Patent Application No. 2013-094079, filed Apr. 26, 2013, theentire contents of each of which is incorporated herein by reference.

TECHNICAL FIELD

The present invention relates to an image acquisition device and amethod and system for acquiring focus information of a sample.

BACKGROUND ART

In an image acquisition device for observing a sample such as a tissuecell, when the distance between the sample on the stage and an objectivelens is kept constant, irregularities on a surface of the sample mayhave an out-of-focus region in an image. Therefore, image acquisitiondevices employing various focusing methods such as a dynamic focusscheme which captures an image of the sample while acquiring focusinformation and a prefocus scheme which acquires focus informationbefore capturing the image of the sample have been developed.

CITATION LIST Patent Literature

Patent Literature 1: Japanese Patent Application Laid-Open No.2012-108184

SUMMARY OF INVENTION Technical Problem

The above-mentioned conventional device can grasp in-focus regions whichare in focus and out-of-focus regions which are out of focus in imagescaptured by an image pickup element. This makes it possible to determinefocal position information from the position of the stage at the timewhen a pixel column corresponding to an in-focus region captures animage. In this technique, however, the position (two-dimensionalposition) of the sample to be subjected to imaging varies among thepixel columns, whereby focus position information of parts slightlydifferent from each other is acquired in practice. The above-mentioneddevice is a microscope device which performs imaging at a highmagnification of 20× to 40×, for example, and thus has such a smalldepth of field that the field of the microscope optical system is verysmall as compared with the sample. Therefore, for acquiring focalposition information of the sample as a whole, it is necessary toperform imaging while moving the field of the microscope optical system,which seems to complicate operations in the device that does not drivethe stage.

For solving the problem mentioned above, it is an object of the presentinvention to provide an image acquisition device and a method and systemfor acquiring focus information of a sample, which can acquire focusinformation of samples rapidly and accurately.

Solution to Problem

For solving the above-mentioned problem, the image acquisition device inaccordance with one aspect of the present invention comprises a stagefor mounting a sample; a light source for emitting light to the sample;a lightguide optical system including an objective lens arranged so asto oppose the sample on the stage; an image pickup element for capturingan optical image of the sample guided by the lightguide optical system;a focus calculation unit for calculating focus information of the sampleaccording to image data from the image pickup element; a first driveunit for moving a field position of the objective lens with respect tothe sample; a second drive unit for changing a focal position of theobjective lens with respect to the sample; and a controller forcontrolling the image pickup element, first drive unit, and second driveunit; the image pickup element is a two-dimensional image pickupelement, adapted to perform rolling readout, having a plurality of pixelcolumns; the controller synchronizes movement of a predetermined part ofthe sample within a field of the objective lens caused by the firstdrive unit with the rolling readout of the image pickup element suchthat each pixel column of the image pickup element is exposed to anoptical image of the predetermined part in the sample, while causing thesecond drive unit to change the focal position of the objective lenswith respect to the sample.

The system for acquiring focus information of a sample in accordancewith one aspect of the present invention comprises a stage for holdingthe sample; a lightguide optical system including an objective lensarranged so as to oppose the sample on the stage; an image pickupelement, constituted by a two-dimensional image pickup element, adaptedto perform rolling readout, having a plurality of pixel columns, forcapturing an optical image of the sample guided by the lightguideoptical system; and a focus calculation unit for calculating focusinformation of the sample according to image data from the image pickupelement; the system synchronizes movement of a predetermined part of thesample within a field of the objective lens with the rolling readout ofthe image pickup element such that each pixel column of the image pickupelement is exposed to an optical image of the predetermined part in thesample, while changing a focal position of the objective lens withrespect to the sample.

The above-mentioned image acquisition device and system use as an imagepickup element a two-dimensional image pickup element which is adaptedto perform rolling readout while having a plurality of pixel columns.The rolling readout scheme, which varies image data readout timingsamong pixel columns and thus may distort images when used for movableobjects, is typically employed for objects which stand still. Incontrast, by utilizing a delay in image data readout timings among pixelcolumns in the rolling readout, the above-mentioned image acquisitiondevice and system synchronize the movement of a predetermined part (thesame part) of the sample within the field of the objective lens with therolling readout such that each pixel column of the image pickup elementis exposed to an optical image of the predetermined part in the sample.As a consequence, image data from each pixel column includes contrastinformation obtained when the focal position of the objective lens ischanged in the same part of the sample, whereby the focus informationcan be calculated rapidly and accurately according to the contrastinformation.

The controller may control the first drive unit such that thepredetermined part of the sample is moved at a fixed speed within thefield of the objective lens. This can easily control the synchronizationof the movement of the predetermined part of the sample within the fieldof the objective lens with the rolling readout.

The controller may start exposing each pixel column of the image pickupelement after a lapse of a predetermined time since the first drive unitstarts moving the field position of the objective lens with respect tothe sample. This can perform exposure favorably.

A plurality of divisional regions where the image pickup elementperforms imaging may be set, while the predetermined part of the samplemay be set so as to be located in a region other than end parts of thedivisional regions. When set in an end part of divisional regions, thepredetermined part of the sample is more susceptible to acceleration atthe time of being moved by the first drive unit. Therefore, setting thepredetermined part of the sample in a region other than end parts of thedivisional regions makes it possible to control the synchronization ofthe movement of the predetermined part of the sample within the field ofthe objective lens with the rolling readout more easily.

The focus calculation unit may calculate the focus information when thefirst drive unit moves the field position of the objective lens betweenthe divisional regions. This can acquire the focus information duringthe movement of the field position of the objective lens, wherebyimaging of the sample can be executed rapidly.

The controller may control the first drive unit such that the fieldposition of the objective lens is moved with respect to the sample overat least three divisional regions. Targeting imaging lines over three ormore divisional regions can shorten the time required for imaging, whilesimplifying control.

The image pickup element may be adapted to switch readout directions ofthe rolling readout. This can extend the degree of freedom in movingdirections of the predetermined part of the sample.

Each pixel column of the image pickup element may be constituted byfirst and second column groups having respective readout directionsdifferent from each other. Such a configuration enables the fieldposition of the objective lens to be bidirectionally scanned in a simplestructure.

The controller may control the second drive unit such that the focalposition of the objective lens with respect to the sample reciprocatesin ascending and descending directions during the synchronization of themovement of the predetermined part of the sample with the rollingreadout of the image pickup element. This can acquire a greater amountof contrast information at the time when the focal position of theobjective lens is changed, whereby the focus information can becalculated more accurately.

The focus calculation unit may produce a focus map according to thecalculated focus information. This can accurately produce the focus map.

The focus calculation unit may calculate the focus information for eachdivisional region. This can accurately produce a focus map of the sampleas a whole.

The focusing method for an image acquisition device in accordance withone aspect of the present invention is a focusing method for an imageacquisition device comprising a stage for mounting a sample; a lightsource for emitting light to the sample; a lightguide optical systemincluding an objective lens arranged so as to oppose the sample on thestage; an image pickup element for capturing an optical image of thesample guided by the lightguide optical system; a focus calculation unitfor calculating focus information of the sample according to image datafrom the image pickup element; a first drive unit for moving a fieldposition of the objective lens with respect to the sample; a seconddrive unit for changing a focal position of the objective lens withrespect to the sample; and a controller for controlling the image pickupelement, first drive unit, and second drive unit; the method comprisingusing as the image pickup element a two-dimensional image pickupelement, adapted to perform rolling readout, having a plurality of pixelcolumns; and causing the controller to synchronize movement of apredetermined part of the sample within a field of the objective lenscaused by the first drive unit with the rolling readout of the imagepickup element such that each pixel column of the image pickup elementis exposed to an optical image of the predetermined part in the sample,while making the second drive unit change the focal position of theobjective lens with respect to the sample.

The method for acquiring focus information of a sample in accordancewith one aspect of the present invention is a method for acquiring focusinformation of a sample by using a two-dimensional image pickup element,adapted to perform rolling readout, having a plurality of pixel columns;the method comprising synchronizing movement of a predetermined part ofthe sample within a field of an objective lens with the rolling readoutof the image pickup element such that each pixel column of the imagepickup element is exposed to an optical image of the predetermined partin the sample, while changing a focal position of the objective lenswith respect to the sample; and acquiring the focus information of thesample according to image data from the image pickup element.

The above-mentioned focusing method for an image acquisition device andmethod for acquiring focus information of a sample use as an imagepickup element a two-dimensional image pickup element which is adaptedto perform rolling readout while having a plurality of pixel columns.The rolling readout scheme, which varies image data readout timingsamong pixel columns and thus distorts images when used for movingobjects, is typically employed for objects which stand still. Incontrast, by utilizing a delay in image data readout timings among pixelcolumns in the rolling readout, the above-mentioned focusing method foran image acquisition device and method for acquiring focus informationof a sample synchronize the movement of a predetermined part (the samepart) of the sample within the field of the objective lens with therolling readout such that each pixel column is exposed to an opticalimage of the predetermined part in the sample. As a consequence, imagedata from each pixel column includes contrast information obtained whenthe focal position of the objective lens is changed in the same part ofthe sample, whereby the focus information can be calculated rapidly andaccurately according to the contrast information.

Advantageous Effects of Invention

The present invention can acquire focus information of samples rapidlyand accurately.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating an embodiment of the image acquisitiondevice in accordance with the present invention;

FIG. 2(a) is an example of an image pickup element in which a pluralityof pixel columns 31, each of which is constructed by arranging aplurality of pixels in a direction perpendicular to a readout direction,align in the readout direction on a light-receiving surface 6 a of theimage pickup element 6. FIG. 2(b) is an example of an image pickupelement in which a reset signal, a readout start signal, and a readoutend signal are outputted according to a drive period of a drive clock,whereby exposure and readout are controlled for each pixel column 31.FIGS. 2(a) and 2(b) represent a light-receiving surface of the imagepickup element and rolling readout in the image pickup element,respectively;

FIG. 3 is a diagram illustrating an example of scanning an imageacquisition region with respect to a sample;

FIG. 4(a) and FIG. 4(b) are diagrams that illustrate how movement of apredetermined part of the sample within a field of an objective lens issynchronized with the rolling readout of an image pickup element 6, inwhich FIG. 4(a) represents the positional relationship between the fieldof the objective lens and divisional regions, while FIG. 4(b) exhibitsthe predetermined part of the sample with respect to each pixel columnand timings at which the image pickup element is exposed and read out;

FIG. 5(a) and FIG. 5(b) are diagrams that illustrate a state subsequentto FIG. 4(a) and FIG. 4(b), respectively;

FIG. 6(a) and FIG. 6(b) are diagrams that illustrate a state subsequentto FIG. 5(a) and FIG. 5(b), respectively;

FIG. 7 is a diagram illustrating an example of pixel columnconfigurations in the image pickup element;

FIG. 8 is a diagram illustrating an example of contrast informationprocessed by an image processing unit;

FIG. 9 is a flowchart illustrating operations of the image acquisitiondevice represented in FIG. 1;

FIG. 10 is a diagram illustrating the relationship between divisionalregions and the predetermined part of the sample when producing a focusmap; and

FIG. 11 is a flowchart illustrating operations of the image acquisitiondevice when producing the focus map.

DESCRIPTION OF EMBODIMENTS

In the following, preferred embodiments of the image acquisition deviceand method and system for acquiring focus information of a sample inaccordance with the present invention will be explained in detail withreference to the drawings.

FIG. 1 is a diagram illustrating an embodiment of the image acquisitiondevice in accordance with the present invention. As illustrated in thediagram, an image acquisition device 1 comprises a stage for mounting asample S, a light source 3 for emitting light to the sample, alightguide optical system 5 including an objective lens 25 arranged soas to oppose the sample S on the stage 2, and an image pickup element 6for capturing an optical image of the sample S guided by the lightguideoptical system 5.

The image acquisition device 1 also comprises a stage drive unit (firstdrive unit) 11 for moving a field position of the objective lens 25 withrespect to the sample S, an objective lens drive unit (second driveunit) 12 for changing the focal position of the objective lens 25 withrespect to the sample S, a controller 13 for controlling the imagepickup element 6, stage drive unit 11, and objective lens drive unit 12,and an image processing unit 14.

The sample S to be observed by the image acquisition device 1, anexample of which is a living sample such as a tissue cell, is mounted onthe stage 2 while being sealed with a glass slide. The light source 3 isarranged on the bottom face side of the stage 2. For example, any oflaser diodes (LD), light-emitting diodes (LED), superluminescent diodes(SLD), and light sources of lamp type such as halogen lamps is used asthe light source 3.

The lightguide optical system 5 is constituted by an illuminationoptical system 21 arranged between the light source 3 and stage 2 and amicroscope optical system 22 arranged between the stage 2 and imagepickup element 6. The illumination optical system 21 has a Köhlerillumination optical system constituted by a condensing lens 23 and aprojection lens 24, for example, and guides the light from the lightsource 3 so as to irradiate the sample S with uniform light. On theother hand, the microscope optical system 22 has an objective lens 25and an imaging lens 26 arranged on the downstream side (image pickupelement 6 side) of the objective lens 25 and guides an optical image ofthe sample S to the image pickup element 6. The optical image of thesample S is an image formed by transmitted light in the case of brightfield illumination, scattered light in the case of dark fieldillumination, and emission (fluorescence) in the case of emissionmeasurement. It may also be an image formed by reflected light from thesample S.

The image pickup element 6 is a two-dimensional image pickup elementwhich is adapted to perform rolling readout while having a plurality ofpixel columns. An example of such an image pickup element 6 is a CMOSimage sensor. As illustrated in FIG. 2(a), a plurality of pixel columns31, each of which is constructed by arranging a plurality of pixels in adirection perpendicular to a readout direction, align in the readoutdirection on a light-receiving surface 6 a of the image pickup element6.

In the image pickup element 6, as illustrated in FIG. 2(b), a resetsignal, a readout start signal, and a readout end signal are outputtedaccording to a drive period of a drive clock, whereby exposure andreadout are controlled for each pixel column 31. An exposure period ofone pixel column 31 is a duration from discharge of electric chargestriggered by the reset signal to readout of the electric chargestriggered by the readout start signal. A readout period of one pixelcolumn 31 is a duration from the start of readout of electric chargestriggered by the readout start signal to an end of readout of electriccharges triggered by the readout end signal. The readout start signalfor the next pixel column can also be used as the readout end signal.

In the rolling readout, readout start signals to be outputted for therespective pixel columns 31 are sequentially outputted with apredetermined time difference. Therefore, unlike global readout in whichall the pixel columns are read out at the same time, respective readoutoperations for the pixel columns 31 are sequentially performed with thepredetermined time difference. The readout speed in the rolling readoutis controlled by a time interval of the readout start signals forreading the respective pixel columns 31. The readout speed becomesfaster and slower as the time interval of readout start signals isshorter and longer, respectively. The readout interval between the pixelcolumns 31, 31 adjacent to each other can be adjusted by techniques suchas adjustment of the frequency of the drive clock, setting of a delayperiod in the readout period, and change of a clock number specifyingthe readout start signal, for example.

The stage drive unit 11 is constituted by a motor or actuator such as astepping motor (pulse motor) or piezoelectric actuator, for example.Under the control of the controller 13, the stage drive unit 11 movesthe stage 2 in the XY directions about a plane having a predeterminedangle (e.g., 90°) with respect to a plane orthogonal to the optical axisof the objective lens 25. As a consequence, the sample S secured to thestage 2 moves relative to the optical axis of the objective lens,thereby shifting the field position of the objective lens 25 withrespect to the sample S.

The image acquisition device 1 performs imaging of the sample at a highmagnification of 20× to 40×, for example. Therefore, the objective lens25 has a field V which is small with respect to the sample S, whereby aregion in which an image can be captured in one imaging operation alsobecomes small with respect to the sample S as illustrated in FIG. 3.This makes it necessary for the field V of the objective lens 25 to bescanned with respect to the sample S in order to capture an image of thesample S as a whole.

Therefore, the image acquisition device 1 sets an image acquisitionregion 32 so as to include the sample S with respect to a samplecontainer (e.g., a glass slide) holding the sample S and configurespositions of a plurality of divisional regions 33 according to the imageacquisition region 32 and the field V on the sample S of the objectivelens 25. Then, an image of a part of the sample S corresponding to thedivisional region 33 is captured, so as to acquire partial image datacorresponding to the divisional region 33, and thereafter the stagedrive unit 11 is driven such that the field V of the objective lens 25is located at the next divisional region 33 to be subjected to imaging,where an image is captured again, so as to acquire partial image data.At this time, the controller 13 drives the stage drive unit 11, so as toaccelerate/decelerate the stage 2 when moving the field V on the sampleS of the objective lens 25 from the divisional region 33 to the nextdivisional region 33 and stop the stage 2 at such a position that thefield V on the sample S is at the next divisional region 33.Subsequently, the image acquisition device 1 repeatedly executes thisoperation, whereupon the image processing unit 14 combines thus obtainedpartial image data, so as to produce an image of the sample S as awhole.

For capturing an image of the sample S as a whole, in the imageacquisition device 1, the controller 13 controls the objective lens 25so as to move its field position with respect to the sample S alongimaging lines Ln (where n is a natural number) constituted by aplurality of divisional regions 33. At this time, the controller 13moves the stage 2 along the scan direction such that the field V on thesample S of the objective lens 25 is located at the next divisionalregion 33 to be subjected to imaging. For moving the field position ofthe objective lens 25 with respect to the sample S between the imaginglines Ln adjacent to each other, bidirectional scanning in which scandirections are reversed between the imaging lines Ln adjacent to eachother, for example, is employed as illustrated in FIG. 3. Unidirectionalscanning in which the same scan direction is used for all the imaginglines Ln may be employed as well. Also employable is random scanning inwhich the field position of the objective lens 25 moves randomly amongthe divisional regions 33.

As with the stage drive unit 11, the objective lens drive unit 12 isconstituted by a motor or actuator such as a stepping motor (pulsemotor) or piezoelectric actuator, for example. Under the control of thecontroller 13, the objective lens drive unit 12 moves the objective lens25 in the Z direction along the optical axis of the objective lens 25.This shifts the focal position of the objective lens 25 with respect tothe sample S.

The controller 13 is a part which controls respective operations of theimage pickup element 6, stage drive unit 11, and objective lens driveunit 12. Specifically, the controller 13 causes the objective lens driveunit 12 to change the focal position (focal plane) of the objective lens25 with respect to the sample S. At this time, the focal position of theobjective lens 25 with respect to the sample S moves along onedirection. Specifically, the controller 13 causes the objective lensdrive unit 12 to change the position in the Z direction of the objectivelens 25 with respect to the stage 2, thereby varying the distancebetween the stage 2 and objective lens 25.

When the stage drive unit 11 can move the stage 2 along the Z directionaligning with the optical axis of the objective lens 25, the controller13 may cause the stage drive unit 11 to change the position in the Zdirection of the stage 2 with respect to the objective lens 25, therebyvarying the distance between the stage 2 and objective lens 25. In thiscase, the stage drive unit 11 serves as a drive unit for moving thefocal position of the objective lens 25 with respect to the sample S,thereby fulfilling functions equivalent to those of this embodiment.

The controller 13 also synchronizes movement of a predetermined part ofthe sample within the field V of the objective lens 25 caused by thestage drive unit 11 with the rolling readout of the image pickup element6 such that each pixel column 31 of the image pickup element 6 isexposed (receives) an optical image of the predetermined part in thesample S. For example, the movement of the stage 2 caused by the stagedrive unit 11 is synchronized with the rolling readout of the imagepickup element 6. When the objective lens drive unit 12 can move thelightguide optical system 5 including the objective lens 25 in the XYdirections, the controller 13 may synchronize the movement of thepredetermined part of the sample within the field V of the objectivelens 25 caused by the objective lens drive unit 12 with the rollingreadout of the image pickup element 6 such that each pixel column 31 ofthe image pickup element 6 is exposed to (receives) the optical image ofthe predetermined part in the sample S. In this case, the objective lensdrive unit 12 serves as a drive unit for moving the field position ofthe objective lens 25 with respect to the sample S, thereby fulfillingfunctions equivalent to those of this embodiment.

This embodiment employs a dynamic prefocus scheme in which focusinformation of the sample S in the next divisional region 33 to besubjected to imaging is acquired immediately before imaging. In thisembodiment, the focus information can be acquired while moving the fieldposition of the objective lens 25 to the next divisional region to besubjected to imaging, whereby the sample S can be placed at an in-focusposition at the time when the field V of the objective lens 25 islocated at the next divisional region 33 to be subjected to imaging.Therefore, the imaging of the sample S can be executed rapidly.

As illustrated in FIG. 4(a), the controller 13 controls the stage driveunit 11 such that the sample S moves at a fixed speed within the field Vof the objective lens 25 when the field V of the objective lens 25shifts from one divisional region 33 a to the next divisional region 33b. As illustrated in FIG. 4(b), the controller 13 also controls thestage drive unit 11 and image pickup element 6 such that the movingdirection of a focused image Sb of the optical image of the sample S onthe light-receiving surface 6 a of the image pickup element 6 and thereadout direction of each pixel column 31 of the image pickup element 6coincide with each other. When an image pickup element which canvariably set the readout speed for the rolling readout is used, thecontroller 13 may change the readout speed for the rolling readoutaccording to the moving speed of the sample S within the field V of theobjective lens 25.

The position of a predetermined part Sa of the sample S used foracquiring focus information can be set according to a time from thestart of movement of the field V of the objective lens 25 with respectto the sample S to the start of exposure of the pixel column 31. Whenthe predetermined part Sa is located at any position in the divisionalregion 33, the controller 13 outputs a reset signal for startingexposure to the image pickup element 6 such as to start the exposure ofthe pixel column 31 after a lapse of a predetermined time since thefield V of the objective lens 25 is moved. When the predetermined partSa is set at a specific position of the divisional region 33, thecontroller 13 calculates the time from the start of movement of thefield V of the objective lens 25 with respect to the sample S to thestart of exposure of the pixel column 31 according to the position ofthe predetermined part Sa in the divisional region 33 and the movingspeed (or acceleration of movement) of the sample S within the field Vof the objective lens 25. Then, the controller 13 starts the exposure ofthe pixel column 31 after a lapse of the calculated time from the startof movement of the field V.

For example, there is a case where it is preferable for thepredetermined part Sa of the sample S to be set in a region other thanthe end parts of the divisional regions 33 (regions abutting onboundaries of the divisional regions 33). In this case, thepredetermined part Sa of the sample S is set so as to keep away from theboundaries of the divisional regions 33, whereby the start of exposureof the pixel column 31 is controlled according to the position of aregion other than the end parts of the divisional region (regionsabutting on the boundaries of the divisional regions 33) and the movingspeed (or acceleration of movement) of the sample S within the field Vof the objective lens 25.

The exposure time in each pixel column 31 is set according to at leastthe width of the predetermined part Sa of the sample S in the scandirection and the moving speed of the predetermined part Sa of thesample S within the field V of the objective lens 25. More preferably,the magnification of the lightguide optical system 5 is also taken intoconsideration. This enables each pixel column 31 to be exposed to theoptical image of the predetermined part Sa of the sample S.

Here, when the focused image Sb of light from the predetermined part Saof the sample S on the light-receiving surface 6 a of the image pickupelement 6 reaches the first pixel column 31 at time T1 as illustrated inFIG. 4(b), the exposure of the first pixel column 31 is started.

At time T2, the position of the sample S within the field V of theobjective lens 25 shifts as illustrated in FIG. 5(a). The focal positionof the objective lens 25 also changes as compared to that at the time T1in such a direction that the gap between the sample S and objective lens25 becomes narrower, for example. The focal position of the objectivelens 25 with respect to the sample S moves along one direction. At thistime, as illustrated in FIG. 5(b), the focused image Sb of light fromthe predetermined part Sa of the sample S reaches the second pixelcolumn 31, thereby starting exposure of the second pixel column 31. Thereadout of the first pixel column 31 is started at the timing when thefocused image Sb of light from the predetermined part Sa of the sample Spasses through the first pixel column 31.

At time T3, the position of the sample S within the field V of theobjective lens 25 further shifts in the scan direction as illustrated inFIG. 6(a). The focal position of the objective lens 25 also changes ascompared to that at the time T1 in such a direction that the gap betweenthe sample S and objective lens 25 becomes narrower. At this time, asillustrated in FIG. 6(b), the focused image Sb of light from thepredetermined part Sa of the sample S reaches the third pixel column 31,thereby starting exposure of the third pixel column 31. The readout ofthe second pixel column 31 is started at the timing when the focusedimage Sb of light from the predetermined part Sa of the sample S passesthrough the second pixel column 31. The readout of the first pixelcolumn 31 ends at the same time when the readout of the second pixelcolumn 31 is started.

Subsequently, the movement of the predetermined part Sa of the sample Swithin the field V of the objective lens 25 and the rolling readout atthe pixel column 31 are performed in the same procedure until apredetermined number of pixel columns is reached. Respective image dataread out from the pixel columns 31 are sequentially outputted to theimage processing unit 14. It is preferable for the image pickup element6 to be able to switch readout directions of the rolling readout. Thismakes it easy for the moving direction of the focused image Sb of lightfrom the sample S and the readout direction of each pixel column 31 ofthe image pickup element 6 to coincide with each other even when thescan direction of the field position of the objective lens 25 withrespect to the sample S changes as in bidirectional scanning and randomscanning.

As illustrated in FIG. 7, a plurality of pixel columns 31 constructingthe light-receiving surface 6 a of the image pickup element 6 may beseparated into first and second pixel column groups 31 a, 31 b, eachconstituted by a plurality of pixel columns, the first and second pixelcolumn groups 31 a, 31 b being read out separately from each other. Inthis case, the readout direction of the first pixel column group 31 aand the readout direction of the second pixel column group 31 b may beset opposite to each other, and the pixel column group used foracquiring focus information may be selected according to the scandirection. Specifically, the first pixel column group 31 a is controlledsuch that pixel groups are sequentially read out from those at an end tothose at the center, and the second pixel column group 31 b is alsocontrolled such that pixel groups are sequentially read out from thoseat an end to those at the center. Of course, the first pixel columngroup 31 a may be controlled such that pixel groups are sequentiallyread out from those at the center to those at an end, and the secondpixel column group 31 b may also be controlled such that pixel groupsare sequentially read out from those at the center to those at an end.This can adapt to bidirectional scanning. The first pixel column group31 a may be controlled such that pixel groups are sequentially read outfrom those at an end to those at the center, while the second pixelcolumn group 31 b may be controlled such that pixel groups aresequentially read out from those at the center to those at an end. Inthis case, the first and second pixel groups 31 a, 31 b are exposed to(receive) optical images of different sample positions, whereby focusinformation can be obtained at two positions at the same time. Inbidirectional scanning, it is preferable for the first and second pixelgroups 31 a, 31 b to have readout directions opposite to each other.

The image processing unit 14 is a part which combines partial image datacaptured by the image pickup element 6, so as to generate an observationimage of the sample S. The image processing unit 14 sequentiallyreceives the respective partial image data of the divisional regions 33outputted from the image pickup element 6 and combines them, so as togenerate an observation image of the sample S as a whole.

The image processing unit 14 also functions as a focus calculation unitwhich calculates focus information of the sample S according to imagedata from the image pickup element 6. Specifically, the image processingunit 14 calculates the focus information of the sample S according tothe respective image data from the pixel columns 31 of the image pickupelement 6. An example of the focus information is such positionalinformation in the Z direction of the objective lens 25 or stage 2 thatthe sample S is located at the focal position of the objective lens 25.Examples of such information include the position in the Z direction ofthe objective lens 25, the height (distance) of the objective lens 25with respect to the sample S (stage 2), the position in the Z directionof the stage 2, and the height (distance) of the sample S (stage 2) withrespect to the objective lens 25.

As mentioned above, the controller 13 synchronizes the movement of thepredetermined part Sa of the sample S within the field V of theobjective lens 25 caused by the stage drive unit 11 with the rollingreadout of the image pickup element 6 such that each pixel column 31 ofthe image pickup element 6 is exposed to an optical image of thepredetermined part Sa in the sample S, while causing the objective lensdrive unit 12 to change the focal position of the objective lens 25.Therefore, in order for each pixel column 31 to be exposed to theoptical image of the predetermined part Sa of the sample S, the imagedata from the image pickup element 6 obtained when the focal position isacquired includes contrast information at the time when the focalposition of the objective lens 25 is changed at the predetermined partSa (the same part) of the sample S.

FIG. 8 is a diagram illustrating an example of contrast informationprocessed by the image processing unit. The example illustrated in thediagram represents contrast values of image data from the first pixelcolumn 31 to the n^(th) pixel column 31 in the imaging region, in whichthe contrast value of the image data in the i^(th) pixel column 31 is apeak value. In this case, assuming that the focal position of theobjective lens 25 is an in-focus position when exposing the i^(th) pixelcolumn to the predetermined part Sa of the sample S, the imageprocessing unit 14 generates focus information. As the contrast value,the contrast value in a specific pixel in the pixels included in eachpixel column 31 or an average value of contrast values in part or wholeof the pixels included in each pixel column 31 may be used.

Operations of the above-mentioned image acquisition device 1 will now beexplained. FIG. 9 is a flowchart illustrating the operations of theimage acquisition device.

In the image acquisition device 1 employing the dynamic prefocus scheme,an image of the sample S is captured in one divisional region 33 (stepS01), whereupon the field V of the objective lens 25 moves toward thenext divisional region 33 (step S02). Next, while the focal position ofthe objective lens 25 is changed during the movement of the field V ofthe objective lens 25, the movement of the predetermined part Sa of thesample S within the field V of the objective lens 25 is synchronizedwith the rolling readout of the image pickup element 6 such that eachpixel column 31 of the image pickup element 6 is exposed to an opticalimage of the predetermined part Sa in the sample S after a lapse of apredetermined time (step S03).

After completing the readout of image data at a predetermined pixelcolumn 31, focus information is calculated according to respectivecontrast values of image data in the pixel columns (step S04). Then, thefocal position of the objective lens 25 with respect to the sample S isadjusted according to the calculated focus information, and an image ofthe sample S is captured in the next divisional region 33 after thefield V of the objective lens 25 is located there (step S05). Theprocessing of steps S01 to S05 is repeated until the imaging of thesample S is completed in all the divisional regions 33, whereupon therespective images of the divisional regions 33 are combined, so as togenerate an observation image of the sample S as a whole.

As explained in the foregoing, the image acquisition device 1 uses asthe image pickup element 6 a two-dimensional image pickup element whichis adapted to perform rolling readout while having a plurality of pixelcolumns 31. The rolling readout scheme, which varies image data readouttimings among the pixel columns 31 and thus may distort images when usedfor movable objects, is typically employed for objects which standstill. In contrast, by utilizing a delay in image data readout timingsamong the pixel columns 31 in the rolling readout, the image acquisitiondevice 1 synchronizes the movement of the predetermined part Sa (thesame part) of the sample S within the field V of the objective lens 25with the rolling readout such that each pixel column 31 of the imagepickup element 6 is exposed to an optical image of the predeterminedpart Sa in the sample S, while changing the focal position of theobjective lens 25. As a consequence, image data from each pixel column31 includes contrast information obtained when the focal position of theobjective lens 25 is changed in the same part of the sample S, wherebythe focus information can be calculated rapidly and accurately accordingto the contrast information.

The image acquisition device 1 also controls the stage drive unit 11such that the predetermined part Sa of the sample S is moved at a fixedspeed within the field V of the objective lens 25. This can easilycontrol the synchronization of the movement of the predetermined part Saof the sample S within the field V of the objective lens 25 with therolling readout.

The predetermined part Sa of the sample S is set so as to be located ina region other than end parts of the divisional regions 33 in the imageacquisition region 32 of the image pickup element 6. When set in an endpart of the divisional regions 32, the predetermined part Sa of thesample S is more susceptible to acceleration at the time of being movedby the stage drive unit 11. Therefore, setting the predetermined part Saof the sample S in a region other than end parts of the divisionalregions 33 makes it possible to control the synchronization of themovement of the predetermined part Sa of the sample S within the field Vof the objective lens 25 with the rolling readout more easily.

The present invention is not limited to the above-mentioned embodiment.For example, while the above-mentioned embodiment assumes to control theobjective lens drive unit 12 so as to move the focal position of theobjective lens 25 in one of ascending and descending directions duringthe synchronization of the predetermined part Sa of the sample S withthe rolling readout of the image pickup element 6, the objective lensdrive unit 12 may be controlled so as to reciprocate the focal positionof the objective lens in ascending and descending directions. In thiscase, the distance (gap) in the Z direction between the objective lens25 and stage 2 is controlled so as to expand and contract repeatedly.

When the sample S is a tissue cell, its thickness is about 10 μm, forexample. Therefore, when the moving distance of the focal position ofthe objective lens 25 for each pixel column 31 is set to about 0.1 μm,contrast information can be acquired for the total thickness of thesample S by about 100 pixel columns. In contrast, a two-dimensionalimage pickup element such as a CMOS image sensor has about severalthousands of pixel columns, for example, whereby contrast informationcan be acquired a plurality of times during one frame. Consequently, byreciprocating the objective lens 25 in the height direction, focusinformation can be calculated for a plurality of predetermined regionsof the sample S, which makes it possible to calculate focus informationmore accurately.

Though the above-mentioned embodiment uses the dynamic prefocus scheme,a focus map scheme can also be employed. While the dynamic prefocusscheme acquires focus information between imaging of one divisionalregion 33 and imaging of the next divisional region 33, the focus mapscheme acquires focus information in each divisional region 33 for theimage acquisition region 32 or imaging line Ln before capturing an imageof the sample S.

In this case, the controller 13 controls the stage drive unit 11 suchthat the field V of the objective lens 25 moves at a fixed speed over aplurality of divisional regions 33. At this time, the moving directionof the focused image Sb of light from a predetermined position of thesample S and the readout direction of the image pickup element 6 aremade to coincide with each other on the light-receiving surface 6 a ofthe image pickup element 6. At the timing when the readout of one frameends, the readout of the next frame is started, which enables thepredetermined part Sa of the sample S used for calculating the focusinformation to appear at fixed intervals, whereby at least one piece offocus information can be acquired in each divisional region 33. A focusmap of the imaging line Ln or the sample S as a whole can be producedaccurately by applying the method of least squares or the like to thefocus information in each divisional region 33.

FIG. 11 is a flowchart illustrating operations of the image acquisitiondevice when producing a focus map. In the image acquisition device 1employing the focus map scheme, as illustrated the chart, the stagedrive unit 11 starts moving the stage 2, whereupon the field V of theobjective lens 25 moves over a plurality of divisional regions 33 (stepS11). It also synchronizes the movement of the predetermined part Sa ofthe sample S with the rolling readout of the image pickup element 6 suchthat each pixel column 31 of the image pickup element 6 is exposed tothe focused image Sb of light of the predetermined part Sa in the sampleS, while changing the focal position of the objective lens 25 so as toreciprocate it along the Z direction (step S12), and calculates focusinformation in each divisional region 33 (step S13).

Thereafter, it is determined whether or not the calculation of focusinformation is completely acquired for a desirable imaging line Ln (stepS14); when the calculation of focus information is not completelyacquired, the field V of the objective lens 25 is moved to the nextimaging line Ln (step S15), and the processing of steps S01 to S03 isrepeatedly executed. When the calculation of focus information iscompletely acquired, a focus map of the sample S is produced accordingto the focus information (step S16). Then, the image data of eachdivisional region 33 is acquired while locating the focal position ofthe objective lens 25 at the sample S according to the produced focusmap.

After focus information for one imaging line Ln is acquired, a focus mapfor this imaging line Ln may be produced, and the respective image dataof the divisional regions 33 constituting the imaging line Ln may beacquired while locating the focal position of the objective lens 25 atthe sample S according to the produced focus map. The focus map may beconstituted by the focus information in each divisional region 33 itselfinstead of being produced by applying the method of least squares or thelike to the focus information in each divisional region 33.

REFERENCE SIGNS LIST

1: image acquisition device; 2: stage; 3: light source; 5: lightguideoptical system; 6: image pickup element; 11: stage drive unit (firstdrive unit); 12: objective lens drive unit (second drive unit); 13:controller, 14: image processing unit (focus calculation unit); 25:objective lens; 31: pixel column; 32: image acquisition region; 33:divisional region; S: sample; Sa: predetermined part of the sample; V:objective lens field.

The invention claimed is:
 1. An apparatus for capturing an imagecomprising: a stage configured to support a sample; a light sourceconfigured to emit light to the sample; an objective lens configured toface to the sample; a two-dimensional image sensor including a pluralityof pixel columns and configured to capture an optical image of thesample and perform rolling readout of an image sensor, wherein the imagesensor comprises a plurality of pixel columns arranged in a directionperpendicular to a readout direction; one or more processors configuredto perform operations comprising: calculating focus information of thesample according to image data from the image sensor; and a firstactuator configured to move a field position of the objective lens withrespect to the sample; a second actuator configured to change a focalposition of the objective lens with respect to the sample; and acontroller configured to: control the image sensor, the first actuator,and the second actuator; synchronize movement of a predetermined part ofthe sample within a field of the objective lens caused by the firstactuator with the rolling readout of the image sensor such that each ofthe plurality of pixel columns of the image sensor is sequentiallyexposed to an optical image of the predetermined part in the sample witha predetermined time difference to acquire image data as each line ofthe optical image is scanned to adjust focus, while causing the secondactuator to change the focal position of the objective lens with respectto the sample, such that an exposure period for one of the plurality ofpixel columns of the image sensor overlaps with a portion of an exposureperiod for a second of the plurality of pixel columns of the imagesensor based upon the predetermined time difference; and wherein amoving speed of the predetermined part of the sample within the field ofthe objective lens is synchronized with a rolling readout speed of theimage sensor.
 2. The apparatus according to claim 1, wherein thecontroller is configured to start exposing each pixel column of theimage sensor after a lapse of the predetermined time since the firstactuator starts moving the field position of the objective lens withrespect to the sample.
 3. The apparatus according to claim 1, wherein aplurality of divisional regions where the image sensor performs imagingare set; and wherein the predetermined part of the sample is set so asto be located in a region other than end parts of the divisionalregions.
 4. The apparatus according to claim 3, wherein the calculatoris configured to calculate the focus information when the first actuatormoves the field position of the objective lens between the divisionalregions.
 5. The apparatus according to claim 4, wherein the calculatoris configured to calculate the focus information for each of thedivisional regions.
 6. The apparatus according to claim 3, wherein thecontroller is configured to control the first actuator such that thefield position of the objective lens is moved with respect to the sampleover at least three divisional regions.
 7. The apparatus according toclaim 1, wherein the image sensor is adapted to switch readoutdirections of the rolling readout.
 8. The apparatus according to claim1, wherein each pixel column of the image sensor is constituted by firstand second column groups having respective readout directions differentfrom each other.
 9. The apparatus according to claim 1, wherein thecontroller is configured to control the second actuator such that thefocal position of the objective lens with respect to the samplereciprocates in ascending and descending directions during thesynchronization of the movement of the predetermined part of the samplewith the rolling readout of the image sensor.
 10. The apparatusaccording to claim 1, wherein the calculator is configured to generate afocus map according to the calculated focus information.
 11. A methodfor acquiring focus information of a sample by using a two-dimensionalimage sensor, adapted to perform rolling readout, including a pluralityof pixel columns; the method comprising: calculating focus informationof the sample according to image data from the image sensor; to adjustfocus, synchronizing movement of a predetermined part of the samplewithin a field of an objective lens with the rolling readout of theimage sensor such that each of the plurality of pixel columns of theimage sensor is sequentially exposed to an optical image of thepredetermined part in the sample with a predetermined time difference toacquire image data as each line of the optical image is scanned, whilechanging a focal position of the objective lens with respect to thesample, such that an exposure period for one of the plurality of pixelcolumns of the image sensor overlaps with a portion of an exposureperiod for a second of the plurality of pixel columns of the imagesensor based upon the predetermined time difference; and acquiring thefocus information of the sample according to image data from the imagesensor; wherein a moving speed of the predetermined part of the samplewithin the field of the objective lens is synchronized with a rollingreadout speed of the image sensor.
 12. The method according to claim 11,wherein the synchronizing starts exposing each pixel column of the imagesensor after a lapse of the predetermined time since the first actuatorstarts moving the field position of the objective lens with respect tothe sample.
 13. The method according to claim 11, wherein a plurality ofdivisional regions where the image sensor performs imaging are set; andwherein the predetermined part of the sample is set so as to be locatedin a region other than end parts of the divisional regions.
 14. Themethod according to claim 13, wherein the calculating calculates thefocus information when the first actuator moves the field position ofthe objective lens between the divisional regions.
 15. The methodaccording to claim 14, wherein the calculating calculates the focusinformation for each of the divisional regions.
 16. The method accordingto claim 13, wherein the synchronizing controls the first actuator suchthat the field position of the objective lens is moved with respect tothe sample over at least three divisional regions.
 17. The methodaccording to claim 11, wherein the image sensor is adapted to switchreadout directions of the rolling readout.
 18. The method according toclaim 11, wherein each pixel column of the image sensor is constitutedby first and second column groups having respective readout directionsdifferent from each other.
 19. The method according to claim 11, whereinthe synchronizing controls the second actuator such that the focalposition of the objective lens with respect to the sample reciprocatesin ascending and descending directions during the synchronization of themovement of the predetermined part of the sample with the rollingreadout of the image sensor.
 20. The method according to claim 11,further comprising: generating a focus map according to the calculatedfocus information.
 21. A system for acquiring focus informationcomprising: a stage configured to support a sample; an objective lensconfigured to face to the sample; a two-dimensional image sensorincluding a plurality of pixel columns and configured to perform rollingreadout and capture an optical image of the sample and perform rollingreadout of an image sensor, wherein the image sensor comprises aplurality of pixel columns arranged in a direction perpendicular to areadout direction; and one or more processors configured to performoperations comprising: calculating focus information of the sampleaccording to image data from the image sensor; and wherein to adjustfocus the system synchronizes movement of a predetermined part of thesample within a field of the objective lens with the rolling readout ofthe image sensor such that each of the plurality of pixel columns columnof the image sensor is sequentially exposed to an optical image of thepredetermined part in the sample with a predetermined time difference toacquire image data as each line of the optical image is scanned, whilechanging a focal position of the objective lens with respect to thesample, such that an exposure period for one of the plurality of pixelcolumns of the image sensor overlaps with a portion of an exposureperiod for a second of the plurality of pixel columns of the imagesensor based upon the predetermined time difference; and wherein amoving speed of the predetermined part of the sample within the field ofthe objective lens is synchronized with a rolling readout speed of theimage sensor.
 22. The system according to claim 21, wherein thecontroller is configured to start exposing each pixel column of theimage sensor after a lapse of the predetermined time since the firstactuator starts moving the field position of the objective lens withrespect to the sample.
 23. The system according to claim 21, wherein aplurality of divisional regions where the image sensor performs imagingare set; and wherein the predetermined part of the sample is set so asto be located in a region other than end parts of the divisionalregions.
 24. The system according to claim 23, wherein the calculator isconfigured to calculate the focus information when the first actuatormoves the field position of the objective lens between the divisionalregions.
 25. The system according to claim 24, wherein the calculator isconfigured to calculate the focus information for each of the divisionalregions.
 26. The system according to claim 23, wherein the controller isconfigured to control the first actuator such that the field position ofthe objective lens is moved with respect to the sample over at leastthree divisional regions.
 27. The system according to claim 21, whereinthe image sensor is adapted to switch readout directions of the rollingreadout.
 28. The system according to claim 21, wherein each pixel columnof the image sensor is constituted by first and second column groupshaving respective readout directions different from each other.
 29. Thesystem according to claim 21, wherein the controller is configured tocontrol the second actuator such that the focal position of theobjective lens with respect to the sample reciprocates in ascending anddescending directions during the synchronization of the movement of thepredetermined part of the sample with the rolling readout of the imagesensor.
 30. The system according to claim 21, wherein the calculator isconfigured to generate a focus map according to the calculated focusinformation.